On the capacity of Markov sources over noisy channels

We present an expectation-maximization method for optimizing Markov process transition probabilities to increase the mutual information rate achievable when the Markov process is transmitted over a noisy finitestate machine channel. The method provides a tight lower bound on the achievable information rate of a Markov process over a noisy channel and it is conjectured that it actually maximizes this information rate. The latter statement is supported by empirical evidence (not shown in this paper) obtained through brute-force optimization methods on low-order Markov processes. The proposed expectationmaximization procedure can be used to find tight lower bounds on the capacities of finite-state machine channels (say, partial response channels) or the noisy capacities of constrained (say, run-length limited) sequences, with the bounds becoming arbitrarily tight as the memory-length of the input Markov process approaches infinity. The method links the Arimoto-Blah& algorithm to Shannon’s noise-free entropy maximization by introducing the noisy a4acency matrix. a similar stochastic expectation-maximization for Markov sources, linking the method to a noisy adjacency matrix, whose computation is shown in Section V. Section VI gives a numeric example by computing a lower bound on the capacity of run-length limited sequences over the binary symmetric channel. Section VII concludes the paper. Notation: The superscript T denotes matrix and vector transposition. Random variables are denoted by uppercase letters, while their realizations are denoted by lowercase letters. If a random variable is a member of a random sequence, an index Y?’ is used to denote time, e.g., Xt. A vector of random variables [Xi, Xi+i,. . . , XjlT is shortly denoted by Xi, while its realization is shortly denoted by z{. The letter H is used to denote the entropy, the letter I denotes mutual information, while the letter Z denotes the mutual information rate.